Maplewood
Memorization and Knowledge Injection in Gated LLMs
Pan, Xu, Hahami, Ely, Zhang, Zechen, Sompolinsky, Haim
Large Language Models (LLMs) currently struggle to sequentially add new memories and integrate new knowledge. These limitations contrast with the human ability to continuously learn from new experiences and acquire knowledge throughout life. Most existing approaches add memories either through large context windows or external memory buffers (e.g., Retrieval-Augmented Generation), and studies on knowledge injection rarely test scenarios resembling everyday life events. In this work, we introduce a continual learning framework, Memory Embedded in Gated LLMs (MEGa), which injects event memories directly into the weights of LLMs. Each memory is stored in a dedicated set of gated low-rank weights. During inference, a gating mechanism activates relevant memory weights by matching query embeddings to stored memory embeddings. This enables the model to both recall entire memories and answer related questions. On two datasets - fictional characters and Wikipedia events - MEGa outperforms baseline approaches in mitigating catastrophic forgetting. Our model draws inspiration from the complementary memory system of the human brain.
- Europe > United Kingdom > England > South Yorkshire (0.05)
- South America > Peru (0.04)
- Europe > France > Île-de-France > Paris > Paris (0.04)
- (12 more...)
- Government (1.00)
- Health & Medicine > Therapeutic Area > Neurology (0.93)
- Education > Educational Setting > K-12 Education (0.68)
- Leisure & Entertainment > Sports > Basketball (0.67)
The Mail
As someone who used to commute through Penn Station, I appreciated William Finnegan's piece about the transit hub and its troubles ("The Way Things Work," March 13th). In 2012, I moved from Brooklyn to suburban New Jersey. I work in urban planning; at the time, I described my new town as a planner's paradise, owing to its walkability and to the fact that my journey to my midtown office seemed to be only about ten minutes longer than it had been when I was taking the subway. Over time, however, the extra ten minutes became an additional half hour, thanks to frequent delays caused by downed wires, signal troubles, and stalled trains. Because of this, when my office called us back to work in person, I decided to find a local job.
- North America > United States > California > San Francisco County > San Francisco (0.17)
- North America > United States > Pennsylvania > Allegheny County > Pittsburgh (0.06)
- North America > United States > New York (0.06)
- (3 more...)
- Media > Film (0.34)
- Health & Medicine > Therapeutic Area (0.34)
Artificial Intelligence Comes To The Construction Site: Startup Pillar Technologies Flags Problems Before Disaster Hits
Pillar cofounders Alex Schwarzkopf and Matt Joyal, Forbes 30 Under 30 alumni, developed rugged, wireless devices for construction sites to flag environmental risks like fires or leaks early. In February 2017, AvalonBay's 235-unit development in Maplewood, New Jersey, burned down six weeks before its planned opening. The homebuilder rebuilt the $55 million project from scratch--then searched for new ways to prevent fires, a constant threat when building with woodframe construction. The solution: Pillar Technologies, whose devices monitor temperature, smoke and other signs of pending disaster, and which will soon use artificial intelligence to flag environmental risks even earlier. "It's like medical monitoring for buildings," says Michael Feigin, chief construction officer and executive vice president at AvalonBay.
- North America > United States > New Jersey > Essex County > Maplewood (0.25)
- North America > United States > New York (0.06)
- Information Technology > Artificial Intelligence (1.00)
- Information Technology > Data Science > Data Mining (0.33)
Business Process Reimagined: AI at Work Accenture
Paul Daugherty is Accenture's Chief Technology and Innovation Officer. Over his career, he has worked with thousands of business and government leaders around the globe, helping them apply technology to transform their organizations. He has also been instrumental in evolving Accenture's business to respond to the exponential changes in technology. Daugherty oversees Accenture's technology strategy and innovation architecture, and he leads Accenture's research and development, ventures, advanced technology, and ecosystem groups. He recently founded Accenture's artificial intelligence business and has led Accenture's research into artificial intelligence over many years.
- North America > United States > Michigan (0.10)
- North America > United States > New Jersey > Essex County > Maplewood (0.07)
DisrupTV Episode 104, Featuring Paul Daugherty, Maribel Lopez, Liza Lichtinger
DisrupTV is a weekly Web series with hosts R "Ray" Wang and Vala Afshar. The show airs live at 11:00 a.m. The audience can expect A-list guests, the latest enterprise news, hot startups, insight from influencers, and much more. We broadcast live on Zoom. The live show will be here at 11 AM PT: https://ConstellationR.zoom.us/j/681590807
- North America > United States > Michigan (0.06)
- North America > United States > New Jersey > Essex County > Maplewood (0.05)
- Asia > China > Hong Kong (0.05)